Similarity Perception of Visual objects: A Machine-Learning Approach

Though a traditional assumption in similarity judgment is that people selectively attend to certain features, few studies have explored the actual method of identifying “salient features” in visual stimuli. In this study, we used complex, realistic images to examine whether people selectively process salient features. Stimuli were triads of original and morphed animal face pictures. In the behavioral experiment, participants viewed two original pictures and a morphed composite of the originals and decided which original picture was more similar to the morphed picture. In the computational analysis, we employed Gabor function and wavelets, Gray-Level Cooccurrence Metrics (GLCM) combined with principal component analysis to extract candidate visual features, such as Gabor texture, brightness, size, and contour for the entire face as well as parts of the face. The simulated annealing algorithm was applied to behavioral data to determine possible weight distributions for the candidate features. The analysis suggests that people selectively attend to a few features when comparing visually complex and realistic images. How do people perceive similarity between visual objects? Similarity research has traditionally assumed that people attend to matching and mismatching features selectively (Sloutsky and Fisher, 2004; Tversky, 1977). Although researchers have demonstrated this idea using semantic concepts and their verbal attributes (Lee & Zeinfuse, 2008), few studies have investigated whether people selectively use a small number of features to perceive similarity of complex visual stimuli. In this article, we collected behavioral data from human participants and applied image-processing and machine-learning techniques to identify the visual features that were consistent with similarity judgments from the behavioral data In brief, we present a computational method to find possible weight distributions for candidate features in visual similarity judgment of animal faces. In the behavioral experiment, human participants judged similarity of original and morphed animal pictures. We collected ten original animal face photographs and created five animal pairs (i.e., bear-fox, cow-pig, hipposheep, koala-rat, and lion-horse pair). For each pair, one original picture (i.e., source) was merged with the other original picture (i.e., target) in Morphman 4.0 (2003). Eighteen morphed pictures were generated for each animal pair. Altogether, 90 morphed pictures (5 animal pairs X 18 morphed pictures) and 10 original pictures were used in the experiment (Figure 1). Participants viewed two original pictures of each pair and one morphed picture of the pair and judged which original picture (left or right) was more similar to their morphed composite (for a similar task, see Sloutsky and Fisher, 2004). The proportion of participants selecting the source picture (e.g., the original hippo face for the hippo-sheep pair) was recorded. To obtain a computational analogue of the behavioral data, we first identified 37 potential visual features that our participants might have used for similarity judgment of animal faces. These candidate features included texture difference, relative brightness and size, and contour of the faces. To obtain textural information, we computed Gabor-based textures Figure 1: Sample stimuli